Programmable display apparatus, control method, and program

ABSTRACT

Provided is a programmable display apparatus that enables post-hoc identification of a user who has been permitted to access an application through facial authentication. The programmable display apparatus stores feature data of a face of a user and identification information of the user in association with each other. The programmable display apparatus performs facial authentication based on image data of a user obtained through image capture and on the feature data. The programmable display apparatus permits a user to access the application if the user has been authenticated. The programmable display apparatus stores identification information of the authenticated user and a history indicating a state in which the access has been permitted in association with each other.

FIELD

The present invention relates to a programmable display apparatus, acontrol method for a programmable display apparatus, and a program forcontrolling a programmable display apparatus.

BACKGROUND

Conventionally, a programmable display apparatus that is communicablyconnected to a programmable logic controller (PLC) is known. Forexample, JP 2008-112222A discloses a programmable display apparatus thatincludes an HMI processing unit and a face log processing unit. The facelog processing unit includes a facial image database, a face detectionunit, a shared memory, a face logging unit, a face log control unit, anda face log file. This programmable display apparatus is communicablyconnected to a camera.

When an operator performs a predetermined switch operation on a screendisplayed on the programmable display apparatus, the HMI processing unitinstructs the face log processing unit to execute processing for logginga facial image of the operator. Once the face log processing unit hasreceived this instruction, the face detection unit detects a full-faceimage from among images of the operator captured by the camera, andstores the detected full-face image into a predetermined facial imagedatabase as a file. The face detection unit also writes a path name ofthe file in the facial image database to the shared memory together withinformation related to an operation history, such as the details of theswitch operation, the date and time of face detection, and a screennumber of the screen on which the switch operation was performed. Theface log control unit instructs the face logging unit to store datawritten to the shared memory into the face log file.

Meanwhile, a facial authentication technique is conventionally known.For example, JP 2004-78687A discloses an entrance/exit management systemthat performs facial authentication. This entrance/exit managementsystem manages entrance to and exit from a facility using a facecross-reference apparatus that cross-references whether or not anentering/exiting person is a pre-registered person based on a facialimage of the entering/exiting person. In the entrance/exit managementsystem, a surveillance camera that captures an image of anentering/exiting person is installed in the vicinity of the facecross-reference apparatus. The image from the surveillance camera isrecorded into a recording unit, and transmitted to an entrance/exitmanagement server together with an entrance/exit history, such as theresult of cross-reference performed by the face cross-referenceapparatus.

JP 2011-59194A discloses an image forming apparatus that includes animage capturing device, an operation screen control unit, and a displayunit. The operation screen control unit includes a facial regiondetection unit, a movement determination unit, a facial featureextraction unit, an attribute detection unit, and a display controlunit. The facial region detection unit detects a facial region of a userfrom image data captured by the image capturing device. The movementdetermination unit determines whether or not the user is approaching theimage forming apparatus. If the movement determination unit determinesthat the user is approaching the image forming apparatus, the facialfeature extraction unit extracts features of a face from the facialregion. The attribute detection unit detects an attribute of the userbased on the features of the face extracted by the facial featureextraction unit. The display control unit displays, on the display unitof the image forming apparatus, an operation screen corresponding to theattribute of the user detected by the attribute detection unit.

JP 2008-165353A discloses a surveillance system that monitors a personwho operates a surveillance apparatus. This surveillance system includesan operator surveillance apparatus. The operator surveillance apparatusincludes a camera, a facial image storage unit, and an operationauthority identification unit. The camera captures a face of theoperating person and outputs facial image data. Facial image data forcross-reference is pre-registered in the facial image storage unittogether with a range of operation authorities. The operation authorityidentification unit identifies a range of operation authorities of theoperating person by cross-referencing facial image data of the operatingperson, which is retrieved either periodically or each time an operationis performed, with the facial image data for cross-reference. Theoperation authority identification unit changes an operable range of thesurveillance apparatus in accordance with the range of operationauthorities.

JP 2008-112222A, JP 2004-78687A, JP 2011-59194A, and JP 2008-165353A areexamples of background art.

The programmable display apparatus of JP 2008-112222A can leave theresult of facial authentication as a history, but cannot permit accessto an application using facial authentication. Therefore, JP2008-112222A does not enable recording of a history indicatingtransition to a state in which access to the application has beenpermitted based on permission to access to the application throughfacial authentication. Permission of access to an application usingfacial authentication is neither disclosed nor suggested in JP2004-78687A, JP 2011-59194A, and JP 2008-165353A.

SUMMARY

The invention of the present application has been made in view of theabove problem. It is an object thereof to provide a programmable displayapparatus that can identify a user who has been permitted to access anapplication through facial authentication in a post-hoc manner, acontrol method used in the programmable display apparatus, and a programfor controlling the programmable display apparatus.

A programmable display apparatus according to one aspect of theinvention can control access to an application. The programmable displayapparatus includes; a storage unit that stores feature data of a face ofa user and identification information of the user in association witheach other; an authentication unit that performs facial authenticationbased on image data of a user obtained through image capture and on thefeature data; an access control unit that permits a user to access theapplication if the user has been authenticated; and a management unitthat stores, into the storage unit, identification information of theauthenticated user and a history (history record, history item)indicating transition to a state in which the access is permitted inassociation with each other.

It is preferable that the programmable display apparatus furtherincludes: a display; and a display control unit that displays anoperation screen including an image of an operation button on thedisplay. If an operation for designating (e.g. clicking on) the image ofthe operation button is performed while the user is permitted to accessthe application, the management unit further stores, into the storageunit, the identification information of the user and a history of theoperation corresponding to the operation button in association with eachother.

It is preferable that, if a user has not been authenticated through thefacial authentication, the management unit further stores, into thestorage unit, image data of a face included in image data of the userobtained through the image capture.

It is preferable that the programmable display apparatus furtherincludes an identification unit that identifies a direction of a line ofsight of a face if image data of the face is included in image data of auser obtained through image capture while the user is permitted toaccess the application. The display control unit displays apredetermined image on the display if a predetermined event hasoccurred. The management unit further stores, into the storage unit, adirection of the line of sight at the time of the occurrence of thepredetermined event in association with the event.

It is preferable that the programmable display apparatus furtherincludes a determination unit that determines, based on the imagecapture, whether or not a facial region included in a subject is largerthan a predetermined size. The access control unit permits a user toaccess the application on the condition that the facial region has beendetermined to be larger than the predetermined size. If a plurality ofusers are included in image data obtained through the image capture, themanagement unit stores, into the storage unit, identificationinformation of a user whose facial region has been determined to belarger than the predetermined size from among the plurality of users,and a history indicating that the access has been permitted, inassociation with each other.

It is preferable that the management unit further stores, into thestorage unit, identification information of users other than the userwhose facial region has been determined to be larger than thepredetermined size from among the plurality of users.

It is preferable that the management unit further stores, into thestorage unit, image data of a user who has not been authenticated fromamong the users other than the user whose facial region has beendetermined to be larger than the predetermined size.

A control method according to another aspect of the invention isexecuted on a programmable display apparatus controlling access to anapplication. The control method includes: a step of performing facialauthentication based on image data of a subject obtained through imagecapture and on feature data indicating a feature of a face of a user; astep of permitting a user to access the application if the user has beenidentified through the facial authentication; and a step of storing,into a memory, identification information of the identified user and ahistory indicating permission of the access in association with eachother.

It is preferable that the control method further includes a step ofdisplaying an operation screen including an image of an operation buttonon a display. In the storing step, if an operation for designating theimage of the operation button is performed while the user is permittedto access the application, the identification information of the userand a history of the operation corresponding to the operation button arestored into the memory in association with each other.

It is preferable that, in the storing step, if a user has not beenauthenticated through the facial authentication, image data of a faceincluded in image data of the user obtained through the image capture isstored into the memory.

It is preferable that the control method further includes: a step ofidentifying a direction of a line of sight of a face if image data ofthe face is included in image data of a user obtained through imagecapture while the user is permitted to access the application; a step ofdisplaying a predetermined image on the display if a predetermined eventhas occurred; and a step of storing, into the memory, a direction of theline of sight at the time of the occurrence of the predetermined eventin association with the event.

It is preferable that the control method further includes a step ofdetermining, based on the image capture, whether or not a facial regionincluded in the subject is larger than a predetermined size. In the stepof permitting the access, a user is permitted to access the applicationon the condition that the facial region has been determined to be largerthan the predetermined size. In the storing step, if a plurality ofusers are included in image data obtained through the image capture,identification information of a user whose facial region has beendetermined to be larger than the predetermined size from among theplurality of users, and a history indicating that the access has beenpermitted, are stored into the memory in association with each other.

It is preferable that, in the storing step, identification informationof users other than the user whose facial region has been determined tobe larger than the predetermined size from among the plurality of usersis stored into the memory.

It is preferable that, in the storing step, image data of a user who hasnot been authenticated from among the users other than the user whosefacial region has been determined to be larger than the predeterminedsize is stored into the memory.

A program according to still another aspect of the invention controls aprogrammable display apparatus controlling access to an application. Theprogram causes a processor of the programmable display apparatus toexecute: a step of performing facial authentication based on image dataof a subject obtained through image capture and on feature dataindicating a feature of a face of a user; a step of permitting a user toaccess the application if the user has been identified through thefacial authentication; and a step of storing, into a memory,identification information of the identified user and a historyindicating permission of the access in association with each other.

The invention enables post-hoc identification of a user who has beenpermitted to access the application through facial authentication.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a front view of a display apparatus.

FIGS. 2A and 2B are diagrams illustrating authentication processing ofthe display apparatus.

FIGS. 3A and 3B are diagrams illustrating examples of operation screensafter a user has been authenticated through facial authentication.

FIG. 4 is a diagram illustrating a history of processing of the displayapparatus.

FIG. 5 is a diagram illustrating an example of a hardware configurationof the display apparatus.

FIG. 6 is a functional block diagram showing a functional configurationof the display apparatus.

FIGS. 7A to 7C are diagrams illustrating screens for configuring systemsettings related to facial authentication.

FIG. 8 is a diagram illustrating data stored in a registered image DB.

FIGS. 9A to 9C are diagrams illustrating facial image data of FIG. 8.

FIGS. 10A to 10C show examples of screens corresponding to authenticatedusers, which are different from the examples of FIGS. 3A and 3B.

FIG. 11 shows data D3 that is referred to when displaying user screensof FIGS. 10A to 10C.

FIG. 12 shows history data D5 managed by a management unit.

FIG. 13 shows data D7 stored in an unregistered person DB.

FIGS. 14A and 14B show screens displayed when processing is restrictedby a restriction unit.

FIG. 15 is a flowchart illustrating the flow of processing of thedisplay apparatus.

FIG. 16 is a diagram illustrating data that is stored in the registeredimage DB in place of the data D1 shown in FIG. 8.

FIG. 17 is a diagram illustrating a configuration for detecting aline-of-sight direction of a user.

FIG. 18 shows a state in which a plurality of users are included inimage data of a subject obtained through image capture using a camera.

DETAILED DESCRIPTION

The following describes in detail a programmable display apparatus(hereinafter simply referred to as “display apparatus”) according to anembodiment of the invention with reference to the drawings. It should benoted that elements in the drawings that are identical or equivalent toone another will be given the same reference numeral, and a descriptionthereof will not be repeated.

<A. External Appearance>

FIG. 1 is a front view of a display apparatus 1 according to the presentembodiment. The display apparatus 1 is connected to a PLC (programmablelogic controller) during use, and functions as a human-machine interface(HMI) for the PLC. Referring to FIG. 1, the display apparatus 1 includesoperation keys 16, a camera 17, and a touchscreen 18. The touchscreen 18is composed of a display and a touchscreen panel.

<B. Outline of Processing>

The display apparatus 1 controls access to an application pre-stored inthe display apparatus 1 through authentication using a facial image of auser. Specifically, the display apparatus 1 permits access to theapplication if the user has been authenticated. The following describesprocessing for permitting access to the application (that is to say,“login processing”) as one example. The following also describes“editing processing” and “history recording processing” to outline theother main types of processing executed by the display apparatus 1.

Hereinafter, a “login state” denotes a state in which access to theapplication is permitted, and a “logout state” denotes a state in whichaccess to the application is not permitted. The display apparatus 1grants the right to access the application if access to the applicationis permitted, and cancels the right to access the application if accessto the application is not permitted.

(b1. Login Processing)

FIGS. 2A and 2B are diagrams illustrating authentication processing ofthe display apparatus 1. Referring to FIGS. 2A and 2B, when a personstands in front of the display apparatus 1 (facing the touchscreen 18),facial authentication is started based on image data of a subject (theperson and background) obtained through image capture performed by thecamera 17, and on feature data of faces of users pre-stored in thedisplay apparatus 1. As facial authentication is a conventionally knowntechnique, a detailed description thereof is not repeated herein.

If a user has been authenticated through facial authentication, thedisplay apparatus 1 permits access to the application. That is to say,in the present embodiment, the display apparatus 1 permits anauthenticated user to log in. Specifically, the display apparatus 1makes a transition to a state in which access to data pre-stored in thedisplay apparatus 1 is permitted. In other words, if the user has beenauthenticated through facial authentication (if the authentication hasbeen successful), the state of the display apparatus 1 is switched froma logout state to a login state. Furthermore, upon switching to thelogin state, the display apparatus 1 displays a predetermined userscreen on the display of the touchscreen 18. In this way, the displayapparatus 1 enables login through facial authentication. It should benoted that a user screen is a screen on which a user performs operationsand/or makes confirmations. The user screen may be an operation screenwith operation buttons, or a screen without operation buttons.

(b2. Editing Processing)

FIGS. 3A and 3B are diagrams illustrating examples of user screens aftera user has been authenticated through facial authentication. FIG. 3A isa diagram illustrating a screen that is displayed on the displayapparatus 1 if a user authenticated through facial authentication hasbeen determined to be an administrator who has a higher operationauthority over the application than a general operator (hereinafter,simply “operator”). FIG. 3B is a diagram illustrating a screen that isdisplayed by the display apparatus 1 if a user authenticated throughfacial authentication has been determined to be an operator.

Referring to FIGS. 3A and 3B, once access to the application has beenpermitted through facial authentication, the display apparatus 1displays, on the display, a screen corresponding to the authenticateduser from among a plurality of screens. For example, if a male α is theadministrator, the display apparatus 1 displays a plurality ofselectable objects 801, 802, 803, 804 on the display. On the other hand,if a female β is the operator, the display apparatus 1 does not displaythe objects 802, 803 corresponding to processing that is permitted onlyfor the administrator (or displays them as unselectable objects). Inthis way, the display apparatus 1 can display a user screencorresponding to the authenticated user.

(b3. History Recording Processing)

FIG. 4 is a diagram illustrating a history of processing of the displayapparatus 1. Referring to FIG. 4, if processing based on a predeterminedcommand has been executed, the display apparatus 1 records the time, thecommand, and the name of a user who has caused the display apparatus 1to start the processing (a user who has been authenticated throughfacial authentication) in association with one another as a history.This history includes a history related to login and logout, and anoperation history of the user in a login state.

For example, in one aspect, the display apparatus 1 stores at leastidentification information (name) of the user who has been authenticatedthrough facial authentication, and a history indicating that access tothe application has been permitted (that is to say, transition to alogin state in which the user is logged in, i.e., “Log in”), inassociation with each other. In one aspect, if an operation fordesignating an image of an operation button included in a user screen isperformed in the login state, the display apparatus 1 stores theidentification information of the user who has been authenticatedthrough facial authentication, and a history of the operationcorresponding to the designated operation button (for example, “PushStart” and “Show Graph”), in association with each other.

The user of the display apparatus 1 (for example, an administrator orlater-described maintenance personnel) can confirm the stored history,as shown in FIG. 4, by instructing the display apparatus 1 to displaythe history on the display. That is to say, the user can performpost-hoc authentication of a user who has logged in through facialauthentication.

The following describes a specific configuration of the displayapparatus 1 for realizing the above-described processing, and thedetails of processing other than the above-described processing.

<C. Hardware Configuration>

FIG. 5 is a diagram illustrating an example of a hardware configurationof the display apparatus 1. Referring to FIG. 5, the display apparatus 1includes a CPU (central processing unit) 11 that executes variouscalculations, a ROM (read-only memory) 12, a RAM (random-access memory)13, a flash ROM 14 that stores various programs in a non-volatilemanner, a clock 15, the operation keys 16, the camera 17, thetouchscreen 18, and a communication interface 19. These elements areconnected to one another via an internal bus.

The touchscreen 18 includes a display 81 and a touchscreen panel 82 thatis arranged to cover the display 81. The communication interface 19includes an Ethernet (registered trademark) IF (interface) 91, a serialIF 92, and a USB (universal serial bus) IF 93.

The CPU 11 deploys the programs stored in the flash ROM 14 into the RAM13 and the like, and executes the deployed programs. The ROM 12generally stores programs such as an operating system (OS). The RAM 13is a volatile memory and used as a working memory.

The Ethernet IF 91 supports Ethernet communication protocols andperforms data communication with the PLC. The serial IF 92 supportsserial communication protocols and performs data communication with, forexample, a PC (personal computer). The USB IF 93 supports USBcommunication protocols and performs data communication with, forexample, a USB memory.

The constituent elements of the display apparatus 1 shown in FIG. 5 arecommon. Therefore, it can be said that the essential part of theinvention is software stored in a memory such as the flash ROM 14, orsoftware that can be downloaded over a network. As the operations ofhardware items of the display apparatus 1 are widely known, a detaileddescription thereof is not repeated.

<D. Details of Processing>

FIG. 6 is a functional block diagram showing a functional configurationof the display apparatus 1. Referring to FIG. 6, the display apparatus 1includes an image capturing unit 101, a size determination unit 102, ageneration unit 103, a facial authentication unit 104, an access controlunit 105, a management unit 106, an input unit 107, a receiving unit108, a display control unit 109, a display unit 110, a character stringdetermination unit 111, a communication processing unit 112, arestriction unit 113, a line-of-sight identification unit 114, and astorage unit 115.

The image capturing unit 101 corresponds to the camera 17 shown e.g. inFIG. 5. The input unit 107 corresponds to the touchscreen panel 82. Thedisplay unit 110 corresponds to the display 81. The storage unit 115corresponds to the flash ROM 14 and the ROM 12.

The storage unit 115 includes a registered image DB (database) 351, aline-of-sight DB 352, an unregistered person DB 353, a history DB 354,an ID (identification)/password DB 355, and a screen data DB 356.

Identification information of a user, facial image data of a face of theuser, feature data of the face of the user, and identificationinformation indicating an operation authority over the application (forexample, administrator, operator, etc.) are registered in the registeredimage DB 351 in association with one another. This set of informationand data is registered in plurality for a plurality of persons.Specifically, identification information indicating an operationauthority is stored in association with the individual pieces of featuredata. It should be noted that feature data is generated from facialimage data by executing a predetermined application program. As oneexample, a facial image is registered based on image capture performedusing the camera 17.

If a user failed to be authenticated through facial authentication,image data of a face of this person who failed to be authenticated isrecorded in the unregistered person DB 353. Various histories of a user(a login history, a logout history, and various operation histories) arerecorded in the history DB 354. Screen data for displaying variousscreens (a user screen and a later-described system setting screen) onthe display 81 of the display apparatus 1 is recorded in the screen dataDB 356.

Information indicating a direction of a line of sight of a user of thedisplay apparatus 1 is recorded in the line-of-sight DB 352. Aconfiguration involving the use of this line-of-sight DB 352 will bedescribed later (FIG. 16). An ID and a password of a user of the displayapparatus 1 are recorded in the ID/password DB 355 on a user-by-userbasis. A configuration involving the use of the ID/password DB 355 willalso be described later.

The image capturing unit 101 captures a subject, and transmits imagedata obtained through this image capture (image data of the subjectincluding facial image data) to the size determination unit 102.Typically, the image capturing unit 101 executes image captureprocessing at designated timings, both in a logout state and in a loginstate. The date and time of image capture are associated with imagedata. To be more precise, the image capturing unit 101 performscontinuous image capture so as to compare facial image data obtained byperforming image capture multiple times with feature data in order toauthenticate a user. For the sake of convenience, image data obtainedthrough this continuous image capture is hereinafter referred to as“frame data”.

The size determination unit 102 determines whether or not a region of aface included in image data is larger than a predetermined size. To beprecise, based on image capture performed by the image capturing unit101, the size determination unit 102 determines whether or not a regionof a face included in a subject is larger than a predetermined size (alater-described minimum value). The size determination unit 102 alsotransmits the result of determination to the generation unit 103 and thefacial authentication unit 104.

The generation unit 103 generates feature data based on facial imagedata in an operation mode for registering facial image data. Thegeneration unit 103 records facial image data and feature data into theregistered image DB in association with each other. Typically, thegeneration unit 103 records facial image data and feature data into theregistered image DB in association with each other if a region of a faceincluded in a subject has been determined to be larger than apredetermined size (a later-described minimum value).

In a logout state (a state in which access to the application is notpermitted), the facial authentication unit 104 automatically performsfacial authentication when a face of a person approaches the imagecapturing unit 101 of the display apparatus 1. In a login state also,the facial authentication unit 104 performs facial authentication uponreceiving a user operation.

Specifically, the facial authentication unit 104 performs authenticationof a user based on image data of the user obtained through image capture(also referred to as “first image data”) and on feature data(hereinafter also referred to as “first facial authentication”). To beprecise, the facial authentication unit 104 performs facialauthentication based on facial image data indicating a face included ina subject and on feature data recorded in the registered image DB. To bemore precise, the facial authentication unit 104 authenticates a user byperforming facial authentication with respect to each one of a pluralityof pieces of frame data.

The facial authentication unit 104 transmits the result of theauthentication to the access control unit 105. If a user failed to beauthenticated, the facial authentication unit 104 transmits image dataof a face that failed to be authenticated (hereinafter also referred toas “unregistered image data”) to the management unit 106.

In addition, at designated timings, the facial authentication unit 104performs authentication of a user based on image data of a face of theuser obtained through image capture in a login state (image captureperformed while access to the application is permitted) (this image datais also referred to as “second image data”), and on feature data(hereinafter also referred to as “second facial authentication”). Theresult of this authentication is transmitted to the restriction unit113.

To be more precise, the facial authentication unit 104 may performfacial authentication with a focus on the eyes. This is because, invenues where the display apparatus 1 is used, there is a highpossibility that parts other than the eyes are covered by clothes.

If a user has been authenticated, the access control unit 105 permitsthe user to access the application. To be precise, the access controlunit 105 permits access to the application on the condition that aregion of a face has been determined to be larger than a predeterminedsize.

Specifically, if a user has been authenticated through facialauthentication in a logout state, the access control unit 105 switchesthe state of the display apparatus 1 from the logout state to a loginstate. To be precise, when permitting access to the application, theaccess control unit 105 lets the display control unit 109 display ascreen (see FIGS. 3A and 3B) on the display 81 (that is to say, thedisplay unit 110) from among a plurality of screens based on featuredata of the authenticated user.

If the access control unit 105 has received information indicating auser with the largest size from the facial authentication unit 104, itdisplays a screen corresponding to this user on the display 81. If thestate of the display apparatus 1 has been switched from the logout stateto the login state, the access control unit 105 transmits informationindicating the transition to the login state to the management unit 106.

To be precise, the access control unit 105 permits access to theapplication (that is to say, switches from the logout state to the loginstate) on the condition that the same user has been authenticated withrespect to at least a predetermined number of pieces of frame data.

The input unit 107 receives various user operations, such as designationof an object and input of character strings. For example, the input unit107 receives a touch operation on a user screen. In this case, the inputunit 107 transmits coordinate values corresponding to the touch input tothe receiving unit 108.

The receiving unit 108 receives the coordinate values from the inputunit 107. The receiving unit 108 judges the input instruction (useroperation), character strings, and the like based on the receivedcoordinate values and on screen information displayed on the displayunit 110. The receiving unit 108 transmits information indicating thisuser operation to the management unit 106. If the receiving unit 108receives an ID and a password as the character strings, it transmits theID and password to the character string determination unit 111.

The display control unit 109 displays a screen on the display 81 (thatis to say, the display unit 110). For example, the display control unit109 displays different screens on the display depending onidentification information indicating the above-described operationauthority (see FIGS. 3A and 3B). To be precise, the display control unit109 displays various screens and the like on the display unit 110 inaccordance with instructions from various elements.

Specifically, the display control unit 109 displays a user screen (e.g.,FIGS. 2A to 3B), a screen for configuring system settings (FIGS. 7A to7C), a history screen (FIG. 4), and the like on the display unit 110.The display control unit 109 also superimposes a predetermined objectimage over these screens displayed. The display control unit 109 alsodisplays various types of information (e.g., numeric values andcharacter strings) transmitted from the PLC on the display unit 110.

The management unit 106 stores identification information of theauthenticated user and a history indicating transition to a state inwhich access is permitted into the storage unit 115 in association witheach other. Specifically, the management unit 106 stores identificationinformation of the authenticated user and a history indicating that theauthenticated user has logged in (“Log in”) into the history DB 354 ofthe storage unit 115 in association with each other. If an operation inwhich an image of an operation button included in a user screen isdesignated while access to the application is permitted (in a loginstate), the management unit 106 stores the identification information ofthe authenticated user and a history of the operation corresponding tothis operation button (e.g., “Push Start” and “Show Graph”) into thehistory DB 354 in association with each other. Any type of historystored in the history DB 354 is associated with date/time informationindicating the date and time of execution of processing or operationcorresponding to the history.

If the management unit 106 receives unregistered image data from thefacial authentication unit 104, it stores the unregistered image datainto the unregistered person DB 353. In this case, date/time informationindicating the date and time of image capture is associated with theunregistered image data.

To be more precise, the management unit 106 stores the above-describedhistories into the history DB 354 on the condition that an administratorhas configured settings for recording histories. In both of a logoutstate and a login state, the management unit 106 may or may not storeimage data of a user who has been authenticated through facialauthenticated into the storage unit 115. One of these modes is adoptedin accordance with a selection by an administrator.

The communication processing unit 112 executes data processing forcommunication with the PLC. The restriction unit 113, the characterstring determination unit 111, and the line-of-sight identification unit114 will be described later.

Below is a more detailed description of the editing processing and thehistory recording processing. Default setting processing and switchdetection will also be described as processing of the display apparatus1. The editing processing is processing in which the display controlunit 109 displays a user screen corresponding to an authenticated userunder control by the access control unit 105 (see FIGS. 3A and 3B). Thehistory recording processing is processing in which the management unit106 stores unregistered image data and a history into the storage unit115.

In the following description, for the sake of convenience, it is assumedthat three people (“administrator”, “maintenance personnel”, and“operator”) who have different ranges of operation authorities (usageauthorities) over the application are defined as users of the displayapparatus 1. These ranges of operation authorities decrease in orderfrom the administrator, the maintenance personnel, to the operator.

(d1. Default Setting Processing)

FIGS. 7A to 7C are diagrams illustrating screens for configuring systemsettings related to facial authentication. FIG. 7A shows a generalsetting screen 511. FIG. 7B shows a facial authentication setting screen512. FIG. 7C shows an advanced setting screen 513. Display of thesescreens for configuring system settings is realized by the displaycontrol unit 109 and the display unit 110.

The display apparatus 1 permits only an administrator to configuresystem settings. That is to say, only a person who is registered in thedisplay apparatus 1 as an administrator can cause display of the screensof FIGS. 7A to 7C.

Referring to FIGS. 7A to 7C, the screens thereof can each be displayedby selecting one of three tabs on the upper part of the screens. Thegeneral setting screen 511 shown in FIG. 7A enables setting of whetheror not to record a history, setting of whether or not to record facialimage data at the time of facial authentication, setting related todetection of switching of people, and setting related to detection of anunregistered person.

The facial authentication setting screen 512 shown in FIG. 7B enablesregistration of a user who needs to be authenticated through facialauthentication, and deletion of the registration of the user. The facialauthentication setting screen 512 also enables registration of aplurality of pieces of facial image data of the same person. If aplurality of pieces of facial image data are registered, a plurality ofpieces of feature data are generated. One of an administrator,maintenance personnel, and operator is set in the “Role” field.

The advanced setting screen 513 shown in FIG. 7C enables setting of aminimum value and a maximum value of a size of a face included in asubject (person and background) at the time of authorizing login throughfacial authentication. The advanced setting screen 513 also enablessetting of a lower limit value (threshold) for a degree of match atwhich the identity of a person is confirmed (likelihood of identity)through facial authentication. As one example, 98% may be set as thethreshold.

FIG. 8 is a diagram illustrating data stored in the registered image DB351. Referring to FIG. 8, a name, a role, a date of update of data,facial image data, and feature data are recorded in data D1 inassociation with one another. As one example, the name “YamashitaTakeshi” is associated with the role “Administrator”, the date of update“2012 Aug. 16”, three pieces of facial image data (“1023KD”, “7544KD”,“9118KD”), and three pieces of feature data (“1023TD”, “7544TD”,“9118TD”). Feature data is generated from facial image data described inthe same field. For example, feature data “1023TD” is generated fromfacial image data “1023KD”.

Up to a predetermined number of pieces of facial image data and featuredata can be registered (for example, ten pieces each). The facialauthentication unit 104 of the display apparatus 1 may be configured todetermine that the authentication has been successful (a user has beenauthenticated) if a degree of match with at least one of the pluralityof pieces of feature data is larger than the set lower limit value(threshold). Alternatively, the facial authentication unit 104 may beconfigured to determine that the authentication has been successful (auser has been authenticated) if the degrees of match with all of theplurality of pieces of feature data are larger than the set lower limitvalue.

FIGS. 9A to 9C are a diagrams illustrating facial image data of FIG. 8.To be precise, FIG. 9A shows an image based on the first facial imagedata “1023KD” of “Yamashita Takeshi”. FIG. 9B is an image based on thesecond facial image data “7544KD” of “Yamashita Takeshi”. FIG. 9C is animage based on the third facial image data “9118KD” of “YamashitaTakeshi”. The accuracy of authentication by the display apparatus 1 canbe improved by registering a plurality of pieces of facial image datapertaining to different states and performing facial authenticationusing these pieces of facial image data in the above-described manner.

(d2. Editing Processing)

As described above, the display apparatus 1 displays a screencorresponding to a user who has been authenticated through facialauthentication. FIGS. 10A to 10C show examples of screens correspondingto authenticated users, which are different from the examples of FIGS.3A and 3B.

FIG. 10A shows a user screen 531 for an administrator. That is to say,FIG. 10A shows a user screen that is displayed if a user authenticatedthrough facial authentication has been pre-registered as anadministrator. FIG. 10B shows a user screen 532 for maintenancepersonnel. That is to say, FIG. 10B shows a user screen that isdisplayed if a user authenticated through facial authentication has beenpre-registered as maintenance personnel. FIG. 10C shows a user screen533 for an operator. That is to say, FIG. 10C shows a user screen thatis displayed if a user authenticated through facial authentication hasbeen pre-registered as an operator.

Referring to FIG. 10A, the user screen 531 includes a status 831, amaintenance menu 832, an admin menu 833, and an object 834 for logout.

Referring to FIG. 10B, the user screen 532 includes the status 831, themaintenance menu 832, and the object 834. Unlike the user screen 531,the user screen 532 does not include the admin menu 833.

Referring to FIG. 10C, the user screen 533 includes the status 831 andthe object 834. Unlike the user screen 531, the user screen 533 does notinclude the admin menu 833 and the maintenance menu 832.

In this way, the display apparatus 1 displays a user screencorresponding to an operation authority based on the result of facialauthentication.

FIG. 11 shows data D3 that is referred to when displaying the userscreens 531, 532, 533 of FIG. 10. Referring to FIG. 11, an object name,coordinate information, and information indicating whether or not todisplay an object image on a role-by-role basis are associated with oneanother in the data D3. For example, the data D3 indicates that threeobject images for a start button, a stop button, and a pause button aredisplayed for an administrator and maintenance personnel. The data D3also indicates that an object image for a system menu is displayed foran administrator, and an object image for logout is displayed for allusers.

The coordinate information is used when displaying an object image onthe display 81. The coordinate information defines coordinates of anupper left portion of the object image, as well as the width and heightof the object image.

(d3. History Recording Processing)

FIG. 12 shows history data D5 managed by the management unit 106. Thatis to say, FIG. 12 shows history data D5 stored in the history DB 354.Referring to FIG. 12, the date and time, a command, and a name arerecorded in the history data D5 in association with one another. If thedisplay apparatus 1 receives an instruction for displaying the historydata D5 from a user (e.g., an administrator), it displays the screenshown in FIG. 4.

FIG. 13 shows data D7 stored in the unregistered person DB 353.Referring to FIG. 13, an image of a person who failed to beauthenticated as a user through facial authentication (an image of anunregistered person) is associated with the date and time of imagecapture in the data D7.

For example, if an operator performs input to the display apparatus 1 toconfirm unregistered people, the display apparatus 1 displays images ofthe unregistered people on the display 81 with reference to the data D7.

(d4. Switch Detection)

A description is now given of switch detection. In the case where thedisplay apparatus 1 is in a login state after a certain person has beenfacially authenticated, the display apparatus 1 may detect a person whois different from the certain person by performing facial authenticationagain. This detection is referred to as switch detection.

As described above, even in a login state, the facial authenticationunit 104 performs facial authentication at designated timings based onimage data of a face obtained through image capture in the login stateand on feature data.

The restriction unit 113 restricts execution of processing of theapplication if a user who has been authenticated through theabove-described first facial authentication is different from a user whohas been authenticated through the above-described second facialauthentication. The restriction unit 113 also restricts execution ofprocessing of the application if a user has not been authenticatedthrough the second facial authentication. Specifically, the restrictionunit 113 restricts execution of predetermined processing that isauthorized in a login state if the result of facial authentication in alogin state does not indicate a user who was authenticated throughfacial authentication in a logout state, or if a user was notauthenticated through facial authentication in a login state.

For example, assume that image capture and authentication are performedfor a non-administrator standing in front of the display apparatus 1while the user screen 531 for an administrator is displayed (FIG. 10A).In this case, the restriction unit 113 restricts at least execution ofprocessing that is allowed in a login state. It is preferable that therestriction unit 113 does not receive selection of an object image thatwas displayed in the login state.

FIGS. 14A and 14B show screens displayed when processing is restrictedby the restriction unit 113. FIG. 14A shows a screen on which warning isdisplayed. FIG. 14B shows a screen after transition from a login stateto a logout state. Whether to display the screen shown in FIG. 14A orthe screen shown in FIG. 14B is set in the display apparatus 1 inadvance. This setting can be changed by an administrator.

Referring to FIG. 14A, when the execution of processing of theapplication is restricted, the access control unit 105 lets the displaycontrol unit 109 display a predetermined object image on the display 81(display unit 110) such that the predetermined object image issuperimposed over a screen that is displayed while access is permitted(in a login state). In this way, the display control unit 109 displays ascreen 551, which is composed of the user screen 531 shown in FIG. 11and a warning object image superimposed thereover, on the display unit110.

Referring to FIG. 14B, the access control unit 105 switches the state ofthe display apparatus 1 from a login state to a logout state.Consequently, when the execution of processing of the application isrestricted, the display control unit 109 displays, on the display 81, ascreen for a state in which access is not permitted in place of a screendisplayed while the access is permitted. Specifically, the displaycontrol unit 109 displays a screen for a logout state on the displayunit 110. In this case, the display apparatus 1 executes image captureand facial authentication processing in a logout state.

While the above is an explanation of an exemplary configuration forcausing transition from a user screen to another screen when processingis restricted by the restriction unit 113, no limitation is intended inthis regard. For example, the display apparatus 1 may restrict onlyprocessing without causing transition of a screen. Settings for thismode are designated by an administrator.

If the result of facial authentication in a login state does notindicate a user who was authenticated through facial authentication in alogout state, the display apparatus 1 may display a user screencorresponding to an operation authority of a user who was authenticatedin the login state. For example, if an operator is authenticated throughfacial authentication in the state of the user screen 531 for anadministrator (FIG. 10A), the display apparatus 1 may display the userscreen 533 for an operator (FIG. 10C). On the other hand, if a differentadministrator is authenticated through facial authentication, the stateof the user screen 531 may be maintained without restricting anoperation authority of the administrator.

<E. Configuration for Control>

FIG. 15 is a flowchart illustrating the flow of processing of thedisplay apparatus 1. Specifically, FIG. 15 shows an aspect of transitionfrom a state in which access to the application is not permitted (alogout state) to a state in which the access is permitted (a loginstate), and then to a state in which the access is not permitted (thelogout state).

Referring to FIG. 15, in step S2, the display apparatus 1 starts imagecapture following an activation. In step S4, the display apparatus 1judges whether or not a face has been detected based on image data of asubject obtained through the image capture. If the display apparatus 1judges that a face has been detected (YES in step S4), it performsfacial authentication in step S6. If the display apparatus 1 judges thatno face has been detected (NO in step S4), processing proceeds to stepS4.

In step S8, the display apparatus 1 judges whether or not a user hasbeen authenticated through facial authentication. If the displayapparatus 1 judges that the user has been authenticated (YES in stepS8), the state of the display apparatus 1 is switched from a logoutstate to a login state in step S10. That is to say, the displayapparatus 1 permits access to the application. If the display apparatus1 judges that the user has failed to be authenticated (NO in step S8),it stores the image data into the unregistered person DB 353 in stepS36.

In step S12, the display apparatus 1 displays a user screencorresponding to the authenticated user. In step S14, the displayapparatus 1 records a history indicating that the authenticated user haslogged in into the history DB 354 in association with the time and thename of the authenticated user. That is to say, the display apparatus 1records a history indicating that access to the application has beenpermitted in association with the time and the name of the authenticateduser. In step S16, the display apparatus 1 judges whether or not anoperation on the user screen has been received. Typically, the displayapparatus 1 judges whether or not an object image has been selected.

If the display apparatus 1 judges that an operation has been received(YES in step S16), it stores a history record of this operation into thehistory DB 354 in association with the time at which this operation wasperformed and the name of the authenticated user in step S18. If thedisplay apparatus 1 judges that no operation has been received (NO instep S16), processing proceeds to step S20.

In step S20, the display apparatus 1 judges whether or not a face hasbeen detected based on image data of a subject obtained through imagecapture. If the display apparatus 1 judges that a face has been detected(YES in step S20), it performs facial authentication in step S22. If thedisplay apparatus 1 judges that no face has been detected (NO in stepS20), processing proceeds to step S20.

In step S24, the display apparatus 1 judges whether or not a user hasbeen authenticated through facial authentication. If the displayapparatus 1 judges that a user has been authenticated (YES in step S24),it judges in step S26 whether or not the authenticated user is the sameperson as the user who was authenticated in the logout state before thelogin. If the display apparatus 1 judges that the user failed to beauthenticated (NO in step S24), it stores the image data into theunregistered person DB 353 in step S38. In step S40, as one example, thedisplay apparatus 1 displays a warning (FIG. 14A).

If the display apparatus 1 judges that the authenticated user is thesame person (YES in step S26), the display apparatus 1 judges in stepS28 whether or not an operation has been received on the user screen. Ifthe display apparatus 1 judges that the authenticated user is not thesame person (NO in step S26), processing proceeds to step S40.

If the display apparatus 1 judges that an operation has been received(YES in step S28), it stores a history record of this operation into thehistory DB 354 in association with the time at which this operation wasperformed and the name of the authenticated user in step S30. If thedisplay apparatus 1 judges that no operation has been received (NO instep S28), processing proceeds to step S32.

In step S32, the display apparatus 1 judges whether or not an operationfor logout has been received. If the display apparatus 1 judges that nooperation for logout has been received (NO in step S32), processingproceeds to step S20. If the display apparatus 1 judges that theoperation for logout has been received (YES in step S32), it stores ahistory record of this operation into the history DB 354 in associationwith the time at which this operation was performed and the name of theauthenticated user in step S34. The display apparatus 1 accordinglyreturns to the logout state, and then ends the processing sequence.

<F. Modification Examples>

(f1. Modification Example of Editing Processing)

The above is an explanation of a configuration in which the displayapparatus 1 displays a screen corresponding to a user who has beenauthenticated through facial authentication. Specifically, the above isan explanation of a configuration in which the display apparatus 1displays a screen based on an operation authority of an authenticateduser (an operation authority of an administrator, an operation authorityof maintenance personnel, and an operation authority of an operator).The following describes a configuration for displaying a screen based onsuch an operation authority and on various types of information relatedto an authenticated user.

FIG. 16 is a diagram illustrating data stored in the registered image DB351 in place of the data D1 shown in FIG. 8. Referring to FIG. 16, aname, a role, a date of update of data, facial image data, feature data,and data defining a display mode (age data, gender data, nationalitydata, and handedness data) are recorded in data D9 in association withone another. The data D9 differs from the data D1 shown in FIG. 8 inincluding data defining a display mode.

The display apparatus 1 displays a screen based on an operationauthority and on data defining a display mode. For example, if a userhas been authenticated as “Yamashita Takeshi” through facialauthentication, a screen for an administrator is displayed such that thelanguage contained in the screen is Japanese. The display apparatus 1also adjusts the size of characters, the brightness of the screen, thecontrast of the screen, and the like to age-appropriate values. At thistime, out of an arrangement of object images (menu and operationbuttons) for left-handedness and an arrangement of object images forright-handedness, the display apparatus 1 displays the screen with thearrangement for right-handedness.

On the other hand, if a user has been authenticated as “Oliver Williams”through facial authentication, a screen for maintenance personnel isdisplayed such that the language contained in the screen is English. Thedisplay apparatus 1 also adjusts the size of characters, the brightnessof the screen, the contrast of the screen, and the like toage-appropriate values. At this time, the display apparatus 1 displaysthe screen with an arrangement for right-handedness. In addition, fromamong a plurality of screens for maintenance personnel, the displayapparatus 1 displays a screen for maintenance personnel that has beenprepared in advance for females.

In this way, in the display apparatus 1, a plurality of pieces offeature data are individually associated with information defining adisplay mode of a screen. The display control unit 109 of the displayapparatus 1 displays a screen corresponding to an authenticated user onthe display unit 110 in a display mode defined by information associatedwith feature data of the authenticated user. Therefore, the displayapparatus 1 can display a screen customized for a user authenticatedthrough facial authentication.

As described above, the display mode includes at least one of thelanguage and the character size. Therefore, the display apparatus 1 candisplay a screen in accordance with the display language and thecharacter size that are considered to be appropriate for theauthenticated user.

Information defining a display mode of a screen includes at least one ofthe age, gender, and nationality. By using such information, the displayapparatus 1 can display a screen appropriate for a user who has beenauthenticated through facial authentication.

(f2. Processing Related to Line of Sight)

FIG. 17 is a diagram illustrating a configuration for detecting aline-of-sight direction of a user. Referring to FIG. 17, upon theoccurrence of a predetermined event in a login state, the displayapparatus 1 displays a predetermined object 891 corresponding to theevent.

If image data of a face is included in image data of a user that wasobtained through image capture while access to the application waspermitted (in a login state), the line-of-sight identification unit 114of the display apparatus 1 (see FIG. 6) identifies a direction of a lineof sight of the face. In this case, the management unit 106 stores adirection of a line of sight at the time of the occurrence of thepredetermined event into the line-of-sight DB 352 of the storage unit115 in association with the event. A direction of a line of sight can beidentified by considering the size of the face (the size of the eyes)included in the captured image data, as well as the positions of irises(or pupils) in the face.

Therefore, by confirming data in the line-of-sight DB 352, anadministrator and the like who has logged in can judge whether or not aperson, e.g., an operator who was working at a venue at the time of theoccurrence of a predetermined event was looking at a screen of thedisplay apparatus 1 at the time of the occurrence of the event.

(f3. Facial Authentication Processing for Plurality of Persons)

The above is an explanation of an exemplary case in which an imageobtained through single image capture shows a single person. Thefollowing describes processing for a case in which an image obtainedthrough single image capture shows a plurality of persons.

FIG. 18 shows a state in which a plurality of users are included inimage data of a subject obtained through image capture using the camera17. Referring to FIG. 18, a person 910 is shown foremost in an image. Aperson 920 and a person 930 are shown in this order behind the person910. Therefore, the size of a facial region decreases in order of theperson 910, the person 920, and the person 930.

If the image capturing unit 101 has transmitted image data that has beenobtained through single image capture and shows a plurality of persons,the size determination unit 102 determines whether or not the people'srespective facial regions are larger than a predetermined size. In thecase of FIG. 18, the size determination unit 102 determines whether ornot the facial regions of the person 910, person 920, and person 930 arelarger than a predetermined size.

If the facial authentication unit 104 receives, from the sizedetermination unit 102, the image data that has been obtained throughsingle image capture and shows a plurality of persons together with theresult of the foregoing determination, the facial authentication unit104 performs facial authentication for the plurality of persons. In thiscase, the facial authentication unit 104 transmits at leastidentification information of one of the authenticated users with thelargest size to the access control unit 105. For example, in the case ofFIG. 18, the facial authentication unit 104 transmits identificationinformation (typically, a name) of the person 910 to the access controlunit 105.

If image data obtained through image capture includes a plurality ofusers, the management unit 106 stores, in association with each other,identification information of those of the users whose facial region hasbeen determined to be larger than a predetermined size (lower limitvalue), and a history record indicating that access to the applicationhas been permitted (that is to say, the user has logged in), into thehistory DB 354 of the storage unit 115. For example, in the case of FIG.18, the management unit 106 stores identification information of theperson 910, together with a history record indicating that the person910 has logged in, into the history DB 354 in association with eachother.

The management unit 106 also stores, into the storage unit 115,identification information of those users other than the user(s) whosefacial region has been determined to be larger than the predeterminedsize. For example, in the case of FIG. 18, the management unit 106stores user information of the person 920 and user information of theperson 930 into the storage unit 115.

In this way, a user of the display apparatus 1 (e.g., an administrator)can identify, from identification information, who was present around aperson who logged in when the state of the display apparatus 1 switchedfrom a logout state to a login state (in the case of FIG. 18, the person910). In the example of FIG. 18, an administrator can identify a name ofthe person 920 and a name of the person 930.

The management unit 106 further stores, into the storage unit 115, imagedata of any not authenticated users among users other than those whosefacial region has been determined to be larger than the predeterminedsize. For example, in the case of FIG. 18, if the facial authenticationunit 104 judges that the person 930 is an unregistered person, themanagement unit 106 stores image data of the person 930 into the storageunit 115.

In this way, a user of the display apparatus 1 (e.g., an administrator)can confirm the face of any unregistered person who was present around aperson who logged in when the state of the display apparatus 1 switchedfrom a logout state to a login state (in the case of FIG. 18, the person910).

(f4. Use of ID and Password)

A description is now given of a configuration involving the use of an IDand a password. The display apparatus 1 may be configured to permit anadministrator and maintenance personnel, who have a broader operationauthority than an operator, to access the application (switch the stateof the display apparatus 1 from a logout state to a login state) on thecondition that the administrator and maintenance personnel have beenauthenticated as users through facial authentication and their IDs andpasswords have matched for the purpose of logging in. The following is adetailed description of this processing.

If a user who has been authenticated through facial authentication hasan operation authority of an administrator or maintenance personnel, thereceiving unit 108 receives input of an ID and a password, which arecharacter strings. The character string determination unit 111determines whether or not the ID and password received by the receivingunit 108 match an ID and a password pre-registered in the ID/password DB355 (predetermined character strings). The access control unit 105switches the state of the display apparatus 1 from a logout state to alogin state on the condition that the character string determinationunit 111 has determined that the ID and password match.

With the foregoing configuration, security can be improved compared to aconfiguration for permitting login only through facial authenticationregardless of a range of operation authorities. While the above is anexplanation of an exemplary configuration in which an administrator andmaintenance personnel are requested to input an ID and a password, aconfiguration for requesting only an administrator to input an ID and apassword may be adopted. Also, the display apparatus 1 may be configuredto request input of only one of an ID and a password.

(f4. Configuration without Built-in Camera)

While the above is an explanation of an exemplary configuration in whichthe display apparatus has a built-in camera, no limitation is intendedin this regard. The display apparatus may be configured to communicatewith an external camera. That is to say, the display apparatus 1 may beconfigured to obtain image data from the external camera.

(f5. Operation Authority)

While the above has exemplarily described an administrator, maintenancepersonnel, and operator who respectively have three different operationauthorities, no limitation is intended in this regard. The number oftypes (categories) of operation authorities may be two, and may be fouror more.

(f6. Application to Apparatus Other than Programmable Display Apparatus)

The above is an explanation of the programmable display apparatus(display apparatus 1) as an example. However, various types ofprocessing described above are applicable not only to the programmabledisplay apparatus, but also to a monitor (display apparatus) includingthe programmable display apparatus.

The embodiment disclosed herein is to be considered in all respects asillustrative, and not restrictive. The scope of the invention isindicated by the claims, rather than by the above description, and isintended to embrace all changes that come within the meaning and scopeof equivalency of the claims.

LIST OF REFERENCE NUMERALS

-   1 display apparatus-   11 CPU-   12 ROM-   13 RAM-   14 flash ROM-   15 clock-   16 operation key-   17 camera-   18 touchscreen-   19 communication interface-   81 display-   82 touchscreen panel-   101 image capturing unit-   102 size determination unit-   103 generation unit-   104 facial authentication unit-   105 access control unit-   106 management unit-   107 input unit-   108 receiving unit-   109 display control unit-   110 display unit-   111 character string determination unit-   112 communication processing unit-   113 restriction unit-   114 line-of-sight identification unit-   115 storage unit-   511 general setting screen-   512 facial authentication setting screen-   513 advanced setting screen-   531, 531, 532, 533, 532, 533 user screen-   801, 802, 803, 804, 834, 891 object-   831 status-   832 maintenance menu-   833 admin menu-   910, 920, 930 person-   351 registered image DB-   352 line-of-sight DB-   353 unregistered person DB-   354 history DB-   355 ID/password DB-   356 screen data DB

The invention claimed is:
 1. A programmable display apparatus capable ofcontrolling access to an application, comprising: a display thatdisplays an operation screen; a storage unit configured to store featuredata of a face of a user and identification information of the user inassociation with each other; an authentication unit configured toperform facial authentication based on image data of a subject obtainedthrough image capture of the subject among a plurality of users, andbased on the pre-stored feature data; and identify the user when thefacial authentication is successful; an access control unit configuredto permit access to the application if the user has been authenticated;a management unit configured to store, into the storage unit,identification information of the authenticated user and a historyindicating transition to a state in which the access is permitted, theidentification information and the history stored in the storage unit inassociation with each other; an identification unit configured toidentify a direction of a line of sight of the face if image data of theface is included in the image data of the subject obtained through theimage capture while the user is permitted to access the application; anda determination unit configured to determine, based on the image captureof the subject among the plurality of users, whether or not a facialregion included in the subject is larger than a predetermined size so asto determine that the subject is foremost in the image data among theplurality of users, wherein the display displays a predetermined imageon the operation screen if a predetermined event has occurred, themanagement unit further stores, into the storage unit, a direction ofthe line of sight at a time that the predetermined event occurred inassociation with the event to confirm that the user was looking at theoperational screen at the time that the predetermined event occurred, ifthe authentication unit receives, from the determination unit, the imagedata that has been obtained through the image capture and the image datashows the plurality of users, the authentication unit performs facialauthentication for each of the plurality of users, and theauthentication unit transmits at least identification information of thesubject from among the plurality of users with the largest size to theaccess control unit, and the access control unit permits the user toaccess the application on a condition that the facial region has beendetermined to be larger than the predetermined size.
 2. The programmabledisplay apparatus according to claim 1, wherein the display displays onthe operation screen, an image of an operation button, wherein if anoperation for designating the image of the operation button is performedwhile the user is permitted to access the application, the managementunit further stores, into the storage unit, the identificationinformation of the user and the history of the operation correspondingto the operation button in association with each other.
 3. Theprogrammable display apparatus according to claim 2, wherein if the userhas not been authenticated through the facial authentication, themanagement unit further stores, into the storage unit, image data of theface included in image data of the subject obtained through the imagecapture.
 4. The programmable display apparatus according to claim 1,wherein if the plurality of users is included in the image data obtainedthrough the image capture, the management unit stores, into the storageunit, identification information of one of the plurality of users whosefacial region has been determined to be larger than the predeterminedsize from among the plurality of users, and the history indicating thatthe access has been permitted, in association with each other.
 5. Theprogrammable display apparatus according to claim 4, wherein themanagement unit further stores, into the storage unit, identificationinformation of users other than the user whose facial region has beendetermined to be larger than the predetermined size from among theplurality of users.
 6. The programmable display apparatus according toclaim 5, wherein the management unit further stores, into the storageunit, image data of the subject who has not been authenticated fromamong the users other than the user whose facial region has beendetermined to be larger than the predetermined size.
 7. A control methodfor controlling a programmable display apparatus controlling access toan application, comprising: displaying an operation screen on a display;performing facial authentication based on image data of a subjectobtained through image capture of the subject including a plurality ofusers, and based on pre-stored feature data indicating a feature of aface of a user; identifying the user when the facial authentication issuccessful; permitting access to the application if the user has beenidentified through the facial authentication; storing, into a memory,identification information of the identified user and a historyindicating permission of the access, the identification information andthe history stored in the memory in association with each other; andidentifying a direction of a line of sight of the face if image data ofthe face is included in the image data of the subject obtained throughthe image capture while the user is permitted to access the application;and determining, based on the image capture of the subject including theplurality of users, whether or not a facial region included in thesubject is larger than a predetermined size so as to determine that thesubject is foremost in the image data among the plurality of users,wherein displaying a predetermined image on the operational screen ofthe display if a predetermined event has occurred, storing, into thememory, a direction of the line of sight at a time that thepredetermined event occurred in association with the event to confirmthat the user was looking at the operational screen at the time that thepredetermined event occurred, if the image data that has been obtainedthrough the image capture shows the plurality of users, the facialauthentication is performed for each of the plurality of users, and atleast identification information of the subject from among the pluralityof users with the largest size is transmitted for permitting access, andpermitting the access to the application if the user has been identifiedthrough the facial authentication comprises permitting the access to theapplication on a condition that the facial region has been determined tobe larger than the predetermined size.
 8. The control method accordingto claim 7, wherein displaying the operation screen comprises displayingan image of an operation button on the display, wherein storing, into amemory, identification information of the identified user and a historyindicating permission of the access comprises storing into the memory,the identification information of the identified user and the historyindicating permission of the access in association with each other, ifan operation for designating the image of the operation button isperformed while the user is permitted to access the application, theidentification information of the user and a history of the operationcorresponding to the operation button are stored into the memory inassociation with each other.
 9. The control method according to claim 8,wherein storing, into a memory, identification information of theidentified user and a history indicating permission of the accesscomprises storing into the memory, image data of the subject, if theuser has not been authenticated through the facial authentication. 10.The control method according to claim 7, wherein storing, into a memory,identification information of the identified user and a historyindicating permission of the access in association with each othercomprises storing into the memory, identification information of theuser whose facial region has been determined to be larger than thepredetermined size from among the plurality of users, and a historyindicating that the access has been permitted, are stored into thememory in association with each other.
 11. The control method accordingto claim 10, wherein storing, into a memory, identification informationof the identified user and a history indicating permission of the accessin association with each other comprises storing into the memory,identification information of users other than the user whose facialregion has been determined to be larger than the predetermined size fromamong the plurality of users is stored into the memory.
 12. The controlmethod according to claim 11, wherein storing, into a memory,identification information of the identified user and a historyindicating permission of the access in association with each othercomprises storing, into the memory, image data of the subject who hasnot been authenticated from among the users other than the user whosefacial region has been determined to be larger than the predeterminedsize is stored into the memory.
 13. A non-transitory processor-readablestorage medium storing a program for controlling a programmable displayapparatus controlling access to an application, the program causing aprocessor of the programmable display apparatus to perform operationscomprising: displaying an operation screen on a display; performingfacial authentication based on image data of a subject obtained throughimage capture of the subject among a plurality of users, and based onstored feature data indicating a feature of a face of a user;identifying the user when the facial authentication is successful;permitting access to the application if the user has been identifiedthrough the facial authentication; storing, into a memory,identification information of the identified user and a historyindicating permission of the access, the identification information andthe history stored in the memory in association with each other;identifying a direction of a line of sight of the face if image data ofthe face is included in the image data of the subject obtained throughthe image capture while the user is permitted to access the application;and determining, based on the image capture of the subject among theplurality of users, whether or not a facial region included in thesubject is larger than a predetermined size so as to determine that thesubject is foremost in the image data among the plurality of users,wherein the program causes the processor of the programmable displayapparatus to perform operations such that: displaying a predeterminedimage on the operational screen of the display if a predetermined eventhas occurred, storing, into the memory, a direction of the line of sightat a time that the predetermined event occurred in association with theevent to confirm that the user was looking at the operational screen atthe time that the predetermined event occurred, if the image data thathas been obtained through the image capture shows the plurality ofusers, the facial authentication is performed for each of the pluralityof users, and at least identification information of the subject fromamong the plurality of users with the largest size is transmitted forpermitting access, and permitting the access to the application if theuser has been identified through the facial authentication comprisespermitting the access to the application on a condition that the facialregion has been determined to be larger than the predetermined size. 14.The non-transitory processor-readable storage medium according to claim13, wherein the program causes the processor of the programmable displayapparatus to perform operations such that: displaying the operationscreen comprises displaying an image of an operation button on adisplay, storing, into a memory, identification information of theidentified user and a history indicating permission of the accesscomprises storing into the memory, the identification information of theidentified user and the history indicating permission of the access inassociation with each other, if an operation for designating the imageof the operation button is performed while the user is permitted toaccess the application, the identification information of the user and ahistory of the operation corresponding to the operation button arestored into the memory in association with each other.
 15. Thenon-transitory processor-readable storage medium according to claim 14,wherein the program causes the processor of the programmable displayapparatus to perform operations such that storing , into a memory,identification information of the identified user and a historyindicating permission of the access comprises storing into the memory,image data of the subject, if the user has not been authenticatedthrough the facial authentication.
 16. The non-transitoryprocessor-readable storage medium according to claim 13, wherein theprogram causes the processor of the programmable display apparatus toperform operations such that: storing, into a memory, identificationinformation of the identified user and a history indicating permissionof the access in association with each other comprises storing into thememory, identification information of the user whose facial region hasbeen determined to be larger than the predetermined size from among theplurality of users, and a history indicating that the access has beenpermitted, are stored into the memory in association with each other.17. The non-transitory processor-readable storage medium according toclaim 16, wherein the program causes the processor of the programmabledisplay apparatus to perform operations such that storing, into amemory, identification information of the identified user and a historyindicating permission of the access in association with each othercomprises storing into the memory, identification information of usersother than the user whose facial region has been determined to be largerthan the predetermined size from among the plurality of users is storedinto the memory.
 18. The non-transitory processor-readable storagemedium according to claim 17, wherein the program causes the processorof the programmable display apparatus to perform operations such thatstoring, into a memory, identification information of the identifieduser and a history indicating permission of the access in associationwith each other comprises storing, into the memory, image data of thesubject who has not been authenticated from among the users other thanthe user whose facial region has been determined to be larger than thepredetermined size is stored into the memory.